Skip to content

perf: skip snapshot rebuild in mergeDuplicates when names are unique - #2441

Merged
zeitlinger merged 2 commits into
prometheus:mainfrom
david-mollitor-db:merge-duplicates-fast-path
Sep 15, 2026
Merged

zeitlinger merged 2 commits into
prometheus:mainfrom
david-mollitor-db:merge-duplicates-fast-path

Conversation

@david-mollitor-db

@david-mollitor-db david-mollitor-db commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What

TextFormatUtil.mergeDuplicates currently only short-circuits when there is a single snapshot. For every scrape with two or more metric families it unconditionally builds a LinkedHashMap, an ArrayList per group, a MetricSnapshots.Builder (with its own list + set) and a freshly sorted MetricSnapshots — even when there is nothing to merge, which is the common case.

Since MetricSnapshots is always sorted by prometheus name (see its constructor), any duplicate names are adjacent. This adds a single allocation-free pass to detect duplicates; when there are none, the input is returned unchanged.

boolean hasDuplicates = false;
for (int i = 1; i < metricSnapshots.size(); i++) {
  if (metricSnapshots.get(i).getMetadata().getPrometheusName()
      .equals(metricSnapshots.get(i - 1).getMetadata().getPrometheusName())) {
    hasDuplicates = true;
    break;
  }
}
if (!hasDuplicates) {
  return metricSnapshots;
}

The merge path for actual duplicates is unchanged.

Why

It removes the map / per-group list / builder / re-sort allocations on every scrape that has no duplicate metric names. Measured on a JMH benchmark of a histogram-heavy scrape (collect + serialize to the Prometheus text format), allocation on the serialize path dropped by ~880 B/op, scaling with the number of metric families.

Correctness

Output is byte-identical. The existing exposition-format tests pass, including DuplicateNamesExpositionTest, which exercises the merge path with real duplicate names.

MetricSnapshots is always sorted by prometheus name, so duplicate names are
adjacent. Detect duplicates in a single allocation-free pass and, when there
are none (the common case), return the input unchanged instead of rebuilding
it through a LinkedHashMap, an ArrayList per group, a MetricSnapshots.Builder
and a freshly sorted MetricSnapshots.

The merge path for actual duplicates is unchanged. Output is byte-identical
(verified by the existing exposition-format tests, including
DuplicateNamesExpositionTest).

Signed-off-by: David Mollitor <david.mollitor@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

Benchmark run succeeded for b7221bcf05690c8a3b6829a1af2026a132ed8c4f.

Prometheus Java Client Benchmarks

Run Information

  • Date: 2026-09-15T12:11:43Z
  • Commit: b7221bc
  • JDK: 25.0.3 (OpenJDK 64-Bit Server VM)
  • Benchmark config: 3 fork(s), 3 warmup, 5 measurement, 4 threads
  • Hardware: AMD EPYC 9V45 96-Core Processor, 4 cores, 16 GB RAM
  • OS: Linux 6.17.0-1022-azure

Comparison with base

  • Head: b7221bc
  • Base: f71915f
  • Metric direction: Throughput scores are higher-is-better; positive Head vs base deltas indicate faster performance.
  • Note: Base and head are compared on the same runner within each topic.
Benchmark PR Base Head vs base Regression verdict
CounterBenchmark.codahaleIncNoLabels 48.65K 49.07K -0.9% within noise
CounterBenchmark.openTelemetryAdd 12.94K 13.02K -0.6% within noise
CounterBenchmark.openTelemetryInc 15.34K 15.00K +2.3% within noise
CounterBenchmark.openTelemetryIncNoLabels 18.45K 18.55K -0.5% within noise
CounterBenchmark.prometheusAdd 51.35K 51.22K +0.3% within noise
CounterBenchmark.prometheusInc 65.61K 65.22K +0.6% within noise
CounterBenchmark.prometheusNoLabelsInc 56.68K 55.33K +2.4% within noise
CounterBenchmark.simpleclientAdd 6.14K 6.23K -1.6% within noise
CounterBenchmark.simpleclientInc 6.51K 6.53K -0.2% within noise
CounterBenchmark.simpleclientNoLabelsInc 6.36K 6.37K -0.1% within noise
HistogramBenchmark.openTelemetryClassic 775.39 777.66 -0.3% within noise
HistogramBenchmark.openTelemetryExponential 878.10 819.76 +7.1% within noise
HistogramBenchmark.prometheusClassic 5.88K 4.85K +21.1% within noise
HistogramBenchmark.prometheusClassicPerThread 11.94K 11.96K -0.2% within noise
HistogramBenchmark.prometheusClassicSingleThread 4.61K 4.62K -0.2% within noise
HistogramBenchmark.prometheusNative 2.76K 2.70K +2.2% within noise
HistogramBenchmark.simpleclient 4.41K 4.42K -0.2% within noise
HistogramTextFormatBenchmark.openMetricsWriteToNull 24.00K 23.28K +3.1% within noise
HistogramTextFormatBenchmark.prometheusWriteToNull 24.23K 24.22K +0.0% within noise
TextFormatUtilBenchmark.openMetricsWriteToByteArray 510.53K 464.77K +9.8% meaningful improvement
TextFormatUtilBenchmark.openMetricsWriteToNull 521.94K 480.10K +8.7% meaningful improvement
TextFormatUtilBenchmark.prometheusWriteToByteArray 536.77K 490.22K +9.5% meaningful improvement
TextFormatUtilBenchmark.prometheusWriteToNull 545.70K 495.01K +10.2% meaningful improvement

Results for PR head

CounterBenchmark

Benchmark Score Error Units
prometheusInc 65.61K ± 555.64 ops/s
prometheusNoLabelsInc 56.68K ± 565.66 ops/s
prometheusAdd 51.35K ± 248.28 ops/s
codahaleIncNoLabels 48.65K ± 2.26K ops/s
openTelemetryIncNoLabels 18.45K ± 179.26 ops/s
openTelemetryInc 15.34K ± 238.89 ops/s
openTelemetryAdd 12.94K ± 76.35 ops/s
simpleclientInc 6.51K ± 50.85 ops/s
simpleclientNoLabelsInc 6.36K ± 27.39 ops/s
simpleclientAdd 6.14K ± 366.04 ops/s

HistogramBenchmark

Benchmark Score Error Units
prometheusClassicPerThread 11.94K ± 21.35 ops/s
prometheusClassic 5.88K ± 937.45 ops/s
prometheusClassicSingleThread 4.61K ± 22.50 ops/s
simpleclient 4.41K ± 81.80 ops/s
prometheusNative 2.76K ± 289.98 ops/s
openTelemetryExponential 878.10 ± 74.54 ops/s
openTelemetryClassic 775.39 ± 15.56 ops/s

HistogramTextFormatBenchmark

Benchmark Score Error Units
prometheusWriteToNull 24.23K ± 549.77 ops/s
openMetricsWriteToNull 24.00K ± 480.29 ops/s

TextFormatUtilBenchmark

Benchmark Score Error Units
prometheusWriteToNull 545.70K ± 14.01K ops/s
prometheusWriteToByteArray 536.77K ± 11.15K ops/s
openMetricsWriteToNull 521.94K ± 13.17K ops/s
openMetricsWriteToByteArray 510.53K ± 7.88K ops/s

Raw Results

Benchmark                                            Mode  Cnt          Score        Error  Units
CounterBenchmark.codahaleIncNoLabels                thrpt   15      48646.756   ± 2255.305  ops/s
CounterBenchmark.openTelemetryAdd                   thrpt   15      12935.486     ± 76.349  ops/s
CounterBenchmark.openTelemetryInc                   thrpt   15      15343.802    ± 238.893  ops/s
CounterBenchmark.openTelemetryIncNoLabels           thrpt   15      18447.882    ± 179.265  ops/s
CounterBenchmark.prometheusAdd                      thrpt   15      51351.820    ± 248.281  ops/s
CounterBenchmark.prometheusInc                      thrpt   15      65605.543    ± 555.641  ops/s
CounterBenchmark.prometheusNoLabelsInc              thrpt   15      56676.074    ± 565.664  ops/s
CounterBenchmark.simpleclientAdd                    thrpt   15       6135.840    ± 366.039  ops/s
CounterBenchmark.simpleclientInc                    thrpt   15       6512.762     ± 50.846  ops/s
CounterBenchmark.simpleclientNoLabelsInc            thrpt   15       6356.790     ± 27.395  ops/s
HistogramBenchmark.openTelemetryClassic             thrpt   15        775.388     ± 15.559  ops/s
HistogramBenchmark.openTelemetryExponential         thrpt   15        878.100     ± 74.539  ops/s
HistogramBenchmark.prometheusClassic                thrpt   15       5875.677    ± 937.445  ops/s
HistogramBenchmark.prometheusClassicPerThread       thrpt   15      11939.216     ± 21.346  ops/s
HistogramBenchmark.prometheusClassicSingleThread    thrpt   15       4610.598     ± 22.498  ops/s
HistogramBenchmark.prometheusNative                 thrpt   15       2757.831    ± 289.980  ops/s
HistogramBenchmark.simpleclient                     thrpt   15       4412.416     ± 81.799  ops/s
HistogramTextFormatBenchmark.openMetricsWriteToNull  thrpt   15      23997.201    ± 480.292  ops/s
HistogramTextFormatBenchmark.prometheusWriteToNull  thrpt   15      24228.358    ± 549.766  ops/s
TextFormatUtilBenchmark.openMetricsWriteToByteArray  thrpt   15     510527.020   ± 7876.057  ops/s
TextFormatUtilBenchmark.openMetricsWriteToNull      thrpt   15     521944.056  ± 13166.783  ops/s
TextFormatUtilBenchmark.prometheusWriteToByteArray  thrpt   15     536773.053  ± 11154.531  ops/s
TextFormatUtilBenchmark.prometheusWriteToNull       thrpt   15     545701.342  ± 14010.015  ops/s

Notes

  • Score = the JMH primary metric; throughput is higher-is-better and latency is lower-is-better.
  • Error = 99.9% confidence interval
  • Regression verdict requires comparable benchmark metadata, non-overlapping JMH confidence intervals, and a change of at least 5%; otherwise it is marked "within noise" or "inconclusive".
  • Scores for different benchmark methods are not ranked against one another; they may measure different workloads.

Benchmark Descriptions

Benchmark Description
CounterBenchmark Counter increment performance: Prometheus, OpenTelemetry, simpleclient, Codahale
HistogramBenchmark Histogram observation performance (classic vs native/exponential)
TextFormatUtilBenchmark Metric exposition format writing speed

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
zeitlinger merged commit 8722230 into prometheus:main Sep 15, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants